1. Technical Field
The present invention relates to image processing apparatuses and the like, and in particular to an image processing apparatus and the like which perform image alignment.
2. Background Art
There are techniques of detecting feature points from an image and techniques of extracting, from the detected feature points, a feature point satisfying a predetermined condition. Hereinafter, detection and extraction of feature points is also referred to simply as feature point extraction. The techniques of feature point extraction are widely used in such areas as image matching, recognition of a particular object in an image, image alignment, and calibration performed when generating a 3D image.
In these areas, a feature point is extracted from each of a plurality of images of an object captured from different viewpoints, and then, matching points are found to make up a pair of feature points corresponding to each other between different images.
More specifically, the matching points are a pair of points representing the same spatial position in different images.
For example, suppose a case of generating a stereoscopic image from two images having a predetermined disparity. Here, the two images have a disparity in the horizontal direction as seen from a viewer, which is necessary for stereoscopic viewing. In addition to the horizontal disparity, there is a possibility for the two images to have a vertical disparity caused by erroneous lens assembly or camera movement, for example. The vertical disparity inhibits comfortable stereoscopic viewing. Thus, it is a common practice to transform one of the images according to the other to reduce the vertical disparity. More specifically, it is necessary to generate a warping matrix for warping one of the images to the other based on a certain condition so that the difference in vertical position between corresponding feature points is reduced.
Thus, for the image alignment, first, feature points are extracted from each of the two images. Next, from the extracted feature points, feature points corresponding to each other between the two images are paired up as a matching pair. After that, a warping matrix is generated in such a manner as to reduce the vertical disparity included in the matching pair. Lastly, one of the images is transformed according to the warping matrix. In this way, a favorable stereoscopic image can be generated.
There are many methods for extracting feature points from an image. For example, various feature point extraction methods are known which use, for instance, Speeded-up Robust Feature (SURF) or Scale Invariant Feature Transform (SIFT) which is a feature amount invariant to image transformation such as rotation and scaling.
For example, U.S. Patent Application Publication No. 2009/0052780 (Patent Literature 1) discloses a technique of dividing an image into a plurality of regions for a multiprocessor system. In Patent Literature 1, Difference of Gaussian (DoG) is used for extracting feature points using SIFT. Here, the total number of feature points extracted from each region is variable and determined according to the threshold of DoG.
Furthermore, U.S. Pat. No. 5,731,851 (Patent Literature 2) discloses a feature-point based motion compensation technique. A region is searched for feature points of a moving object, and a grid associated with those feature points is formed with a hierarchical structure for coding purposes.
Moreover, U.S. Pat. No. 5,617,459 (Patent Literature 3) discloses a method of extracting feature points on the contour of an object.